Overlays Displayed on Top of Media Content
It is a common practice today that when users are viewing media content on their screens at their homes, there are multiple types of overlays displayed on top of the media content they want to watch.
The overlays may be advertisements trying to sell goods or services, which may or may not be related to the contents of the currently presented media. The overlays may be elements of a graphical user interface (GUI) such as pop-up control windows containing buttons the user may activate for selecting viewing modes. The overlays may be subtitles presenting translation of spoken text into a language different from a movie's original language. It is not uncommon to have multiple overlays of different types being simultaneously presented in different screen locations.
When viewing media content such as a movie or a sport competition like football or basketball game there are regions on the screen which are of relatively higher interest to the user than the rest of the screen. For example, the heads of the actors speaking in a movie scene have a high interest level. The same is true of a region in which the ball is currently located in a ball game.
It may happen that an overlay being displayed on top of the media content creates an interference with the media content by fully or partially hiding a region of interest. This might be very disturbing and frustrating to the user. A user might even get so irritated with such interference that he will develop negative feelings against an item advertised in the overlay, an effect that defeats the purpose of presenting the advertisement.
FIGS. 1A-1B each illustrate video content comprising four frames—in the example of FIGS. 1A-1B two cars move towards a corner of the screen (bottom, right corner) where, upon meeting a ‘crash’ is illustrated.
Algorithms are known in the art to electronically detect one or more ‘region(s) of interest’ (ROI) within a frame of media content. In the non-limiting example of FIGS. 1A-1B, two regions (respectively labeled as 120A and 120B) may be identified as ‘regions of interest.’ In video media content the ROIs may move—thus, as shown in FIGS. 1A-1B, first ROI 120A moves horizontally across the screen from left to right and the second ROI 120B moves vertically across the screen from top to bottom. The ROIs merge in “Frame 4” where there is a single merged ROI 120C.
FIG. 2A illustrates interference caused by the overlay to the video content. In Frames 1, 2 and 4 there is no such interference. In frame 3 of FIG. 2A, there is overlap between overlay 130 and second ROI 120B such that the overlay 130 blocks at least a portion of the second ROI 120B.
FIG. 2B is another example. Frames 1, 2 and 4 of FIG. 2B are respectively identical to Frames 1, 2 and 4 of FIG. 2A. Frame 3 of FIG. 2B illustrates interference caused by the overlay 130 to the media content within the second ROI 120B. Thus, in both FIG. 2A and FIG. 2B the overlay causes interference—however, the ‘extent’ of interference in Frame 3 of FIG. 2B is less than the extent of interference in Frame 3 of FIG. 2A.
FIG. 3 illustrates another example of interference by another overlay 130 (in this example a circular overlay) to underlying video content. In the example of FIG. 3, there is no interference in Frames 1, 3 and 4. In contrast, in Frame 3 of FIG. 3, circular overlay 130 causes ‘significant’ interference to the underlying content.
FIG. 4A illustrates frames of another video content item. In FIG. 4A, a semi-circular object moves horizontally from left to right. Also illustrated in FIG. 4A is ROI 120 which moves together with the semi-circle. In the example of FIG. 4A, the semi-circle is bright orange (see ‘Color Legend’ on the right hand side of FIG. 4A).
FIG. 4B is the same as FIG. 4A except for the presence of bright-orange overlay 130 in FIG. 4B. Although overlay 130 does not ‘block’ the semi-circle, it is noted that (i) in all frames, overlay 130 and the semi-circle have the same color, (ii) in Frames 2-3, the lower horizontal edge of rectangular overlay 130 is extremely close to the upper horizontal edge of the semi-circle. In Frames 2-3, the combination of matching colors and the proximity of the horizontal edge means that the presence of the overlay makes it more difficult for a viewer to detect the upper horizontal edge of the semi-circle.
Reference is now made to FIG. 5. As illustrated in FIG. 5, it is noted that for video content, the amount of ‘interference’ can increase or decrease as the video content evolves. In the example of FIG. 5, the extent of interference increases between Frame 1 and Frame 2, decreases between Frame 2 and Frame 3, and decreases between Frame 3 and Frame 4.
As illustrated in FIG. 6 which also relates to video media content (see Frames 1-4), it is possible for underlying media content to have multiple ROIs 120A, 120B and/or multiple overlays 130A, 130B. In frames 1 and 4, there is no interference. In Frame 2 first overlay 130A causes interference to media content in ROI 120A; in Frame 3 second overlay 130B causes interference to media content in ROI 120B.
U.S. Pat. No. 8,451,380 and US Patent Application No. 2011/0188836, each of which is incorporated by reference in its entirety, each disclose changing the screen positions of interfering overlays and also disclose algorithms for distinguishing between safe and unsafe locations for the overlays.
The prior art solution presented above may indeed reduce the level of interference created by overlays, but it has a significant drawback. What the user sees during the time interval in which an overlay such as a GUI window is displayed is a window jumping around the screen as the regions of interest in the background media content evolve with time. It might be very difficult for the user to select a button (for example using a cursor controlled by a mouse) when the button seems to be “running away” from the cursor as it gets close to it. This might be a frustrating experience for the user.
But even with overlays that do not require user input such as simple advertisements the experience of the overlay randomly shifting around might still be frustrating. There is thus a need for a solution that solves the overlay interference problem without suffering from this disturbing effect.
Local Terminals
FIG. 7A illustrates a local terminal 200A (e.g. a mobile phone, a laptop, a desktop, etc) configured to display media content (e.g. remote media content 98 downloaded into local terminal 200A by download module 210) on display screen 210.
Local terminal 200A includes a media-playing module 250 (e.g. comprising a CODEC for decoding content item 98 or a portion thereof—for example, a hardware media decoder) for playing media content, a microprocessor(s) 220 for executing code of local application 240 and a user input device 418.
For local terminals that download their content, a portion of the content (e.g. one or more video frames of video content) may be stored in buffer 260—by storing video frames in buffer 260, these frames are immediately available for display irrespective of the rate of download of media content.
Another example of a local terminal 200B is illustrated in FIG. 7B. Instead of, or in addition to, having an ‘onboard display screen 210’ (as in the example of FIG. 7A), a local terminal may export a visual-content signal (e.g. video signal) via respective ports 280, 310 to an external rendering device 300 so that visual content is displayed on screen 320.
The following issued patents and patent publications provide potentially relevant background material, and are all incorporated by reference in their entirety: U.S. Pat. Nos. 5,953,076, 7,015,978, 8,369,686, 8,451,380, US20110188836 US20110178854, US20110012812, US2013227622 and WO2001072040